Method and apparatus for removing image noise

ABSTRACT

Provided are a method and apparatus for removing image noise. The method includes: separating an input image signal into a signal component and a noise component; converting the noise component into a decorrelated noise component that is spatiotemporally decorrelated from neighboring pixels; and generating an image signal by adding the decorrelated noise component to the signal component.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application is a continuation of U.S. application Ser. No.12/638,515 filed on Dec. 15, 2009, which claims priority from KoreanPatent Application No. 10-2008-0128203, filed on Dec. 16, 2008, in theKorean Intellectual Property Office, the disclosures of which areincorporated herein in its entirety by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and apparatus for processingan image signal, and more particularly, to a method and apparatus forremoving image noise by performing spatiotemporal decorrelation on a lowfrequency noise component of an image signal.

2. Description of the Related Art

In general, noise is unavoidably added to an image signal while theimage signal is acquired by a digital camera and transmitted to adigital television (TV) through a broadcast channel.

The noise deteriorates the quality of an image, thereby making itdifficult to provide a high quality image to viewers.

Accordingly, there is a demand for technology that can remove imagenoise in order to obtain a high quality image.

SUMMARY OF THE INVENTION

The present invention provides a method and apparatus for removing imagenoise by performing spatiotemporal decorrelation on a low frequencynoise component of an image signal.

According to an aspect of the present invention, there is provided amethod of removing image noise, the method comprising: separating aninput image signal into a signal component and a noise component;converting the noise component into a decorrelated noise component thatis spatiotemporally decorrelated from neighboring pixels; and generatingan image signal by adding the decorrelated noise component to the signalcomponent.

According to another aspect of the present invention, there is providedan apparatus for removing image noise, the apparatus comprising: asignal separating unit separating an input image signal into a signalcomponent and a noise component; a decorrelation performing unit whichconverts the noise component into a decorrelated noise component that isspatiotemporally decorrelated from neighboring pixels; and an addingunit which adds the decorrelated noise component to the signal componentto obtain an output image signal.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present inventionwill become more apparent by describing in detail exemplary embodimentsthereof with reference to the attached drawings in which:

FIG. 1 is a block diagram of an apparatus for removing image noise,according to an exemplary embodiment of the present invention;

FIG. 2 is a block diagram of a signal separating unit according to anexemplary embodiment of the present invention;

FIG. 3 illustrates a noise waveform for explaining the principle ofdecorrelation performed by a decorrelation performing unit according toan exemplary embodiment of the present invention;

FIG. 4A is a block diagram of the decorrelation performing unitaccording to an exemplary embodiment of the present invention;

FIG. 4B is a block diagram of the decorrelation performing unitaccording to another exemplary embodiment of the present invention;

FIG. 5A illustrates the noise waveform of a noise component processed bythe decorrelation performing unit according to an exemplary embodimentof the present invention;

FIG. 5B illustrates the noise waveform of a noise component processed bythe decorrelation performing unit according to an exemplary embodimentof the present invention;

FIG. 6 is a block diagram of a motion compensation filtering unitaccording to an exemplary embodiment of the present invention;

FIG. 7 is a flowchart illustrating a method of removing image noise,according to an exemplary embodiment of the present invention;

FIG. 8A is a flowchart illustrating a decorrelated noise signalprocessing process according to an exemplary embodiment of the presentinvention; and

FIG. 8B is a flowchart illustrating the decorrelated noise signalprocessing process according to another exemplary embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention will now be described more fully with reference tothe accompanying drawings, in which exemplary embodiments of theinvention are shown.

FIG. 1 is a block diagram of an apparatus for removing image noise,according to an exemplary embodiment of the present invention.

Referring to FIG. 1, the apparatus includes a signal separating unit110, a decorrelation performing unit 120, an adding unit 130, and amotion compensation filtering unit 140.

The signal separating unit 110 separates an input image signalI_(i)(x,y,t) into a signal component I_(S)(x,y,t) and a noise componentI_(N)(x,y,t). The noise component I_(N)(x,y,t) includes a general randomhigh frequency component and a low frequency component havingspatiotemporal correlation. Noise of the low frequency componentincludes noise, which is produced when a digital image signal iscompressed, and film grain noise, which is produced in a photographicfilm of a film camera.

The decorrelation performing unit 120 converts the noise componentI_(N)(x,y,t), which is obtained by the signal separating unit 110, intoa decorrelated noise component I_(d)(x,y,t) that is spatiotemporallydecorrelated from neighboring pixels. That is, the decorrelationperforming unit 120 converts the noise component I_(N)(x,y,t) into thedecorrelated noise component I_(d)(x,y,t) that is decorrelated fromneighboring pixels in a field (or pixels between a previous field and acurrent field) in such a manner that a noise component of the currentfield is converted into a high frequency random noise component.Optionally, the decorrelation performing unit 120 generates a highfrequency random noise component by selectively outputting the noisecomponent of the current field and a noise component of the previousfield according to motion information and a random signal.

The decorrelation performing unit 120 may perform decorrelationdifferently on a luminance noise component and a chrominance noisecomponent, respectively.

For example, the decorrelation performing unit 120 may convert aluminance noise component of the current field into a high frequencyrandom noise component, and may convert a difference between achrominance noise component of the current field and a chrominance noisecomponent of the previous field into a high frequency random chrominancenoise component.

The adding unit 130 adds the decorrelated noise component I_(d)(x,y,t),which is obtained by the decorrelation performing unit 120, to thesignal component I_(S)(x,y,t), which is obtained by the signalseparating unit 110, to obtain an image signal I_(m)(x,y,t) and outputsthe image signal I_(m)(x,y,t) that is decorrelated from low frequencynoise.

The motion compensation filtering unit 140 performs motion-compensatedlow-pass filtering on the image signal I_(m)(x,y,t) of the currentfield, which is output from the adding unit 130, to obtain an imagesignal I_(o)(x,y,t), and outputs the image signal I_(o)(x,y,t) with nolow frequency noise. The motion compensation filtering unit 140 may bean infinite impulse response (IIR) filter.

FIG. 2 is a block diagram of the signal separating unit 110 of anapparatus, of which an example is shown in FIG. 1.

The signal separating unit 110 includes a low-pass filter 210 and asubtractor 220.

The low-pass filter 210 performs low-pass filtering on the input imagesignal I_(i)(x,y,t) and outputs the signal component I_(S)(x,y,t) withno noise. The input image signal I_(i)(x,y,t) includes a luminancecomponent and/or a chrominance component.

The subtractor 220 extracts the noise component I_(N)(x,y,t) bysubtracting the signal component I_(S)(x,y,t), which is obtained by thelow-pass filter 210, from the image signal I_(i)(x,y,t).

FIG. 3 illustrates a noise waveform for explaining the principle ofdecorrelation performed by the decorrelation performing unit 120 of anapparatus, of which an example is shown in FIG. 1.

In FIG. 3, the x-axis represents a coordinate space, and the y-axisrepresents the level of the noise component.

Referring to FIG. 3, a low frequency noise signal is converted into ahigh frequency random noise signal after noise decorrelation.

FIG. 4A is a block diagram of the decorrelation performing unit 120 ofan apparatus, of which an example is shown in FIG. 1, according to anexemplary embodiment of the present invention.

Referring to FIG. 4A, the decorrelation performing unit 120 includes again control unit 424 and a random signal generating unit 426.

The random signal generating unit 426 generates a random number. Itshould be noted that while the random number in this exemplaryembodiment is described as a binary number, the present invention is notlimited thereto. One of skill in the art would recognize that othertypes of values may be used to perform the operations described herein.

The gain control unit 424 adjusts a gain of the noise componentI_(N)(x,y,t) which is a low frequency noise component, according to therandom binary number, which is generated by the random signal generatingunit 426, and converts the noise component I_(N)(x,y,t) into a highfrequency random noise component.

FIG. 4B is a block diagram of the decorrelation performing unit 120 ofan apparatus, of which an example is shown in FIG. 1, according toanother exemplary embodiment of the present invention.

Referring to FIG. 4B, the decorrelation performing unit 120 includes amotion detecting unit 432, a signal selecting unit 434, and a randomsignal generating unit 436.

The motion detecting unit 432 determines whether there is image motionby using a difference between the image signal I_(i)(x,y,t) of thecurrent field and an image signal I_(i)(x,y,t-1) of the previous field.

The random signal generating unit 436 generates a random binary number.

The signal selecting unit 434 selectively outputs the noise componentI_(N)(x,y,t), e.g., the chrominance noise component, of the currentfield or a noise component I_(N)(x,y,t-1), e.g., a chrominance noisecomponent, of the previous field according to the determination resultof whether there is image motion, which is obtained by the motiondetecting unit 432, and a random binary number signal.

FIG. 5A illustrates the noise waveform of a noise component processed bythe decorrelation performing unit 120, of which an example is shown inFIG. 4A.

In FIG. 5A, the x-axis represents a coordinate space, and the y-axisrepresents the level of the noise component.

Referring to FIG. 5A, a low frequency noise signal 510 in space isconverted into a high frequency random noise signal 520 (marked by “”)through noise decorrelation. That is, a “0” level value and a currentlevel value are randomly generated according to a random binary noisesignal.

FIG. 5B illustrates the noise waveform of a noise component processed bythe decorrelation performing unit 120, of which an example is shown inFIG. 4B.

In FIG. 5B, the x-axis represents a coordinate space, and the y-axisrepresents the level of the noise component.

Referring to FIG. 5B, a low frequency noise signal 530 of a currentfield and noise 540 of a previous field in space are converted into ahigh frequency random noise signal 550 (marked by “”) through noisedecorrelation.

FIG. 6 is a block diagram of the motion compensation filtering unit 140of an apparatus, of which an example is shown in FIG. 1.

The motion compensation filtering unit 140 includes a motion estimatingunit 610 and a low-pass filter 620.

The motion estimating unit 610 estimates motion by using temporalcorrelation between adjacent fields or frame images. For example, themotion estimating unit 610 estimates a motion vector MV for each blockimage by calculating a difference between a block image I_(o)(x,y,t−1)of a reference field (or the previous field) and a block imageI_(m)(x,y,t−1) of the current field.

The low-pass filter 620 performs motion-compensated low-pass filteringon the image signal I_(m)(x,y,t), which is output from the adding unit130, based on the motion vector MV that is obtained by the motionestimating unit 610. The low-pass filter 620 may be an IIR filter havinga feedback loop.

FIG. 7 is a flowchart illustrating a method of removing image noise,according to an exemplary embodiment of the present invention.

In operation 710, an image is input.

In operation 720, the input image is separated into a noise componentand a signal component by using a low-pass filter.

In operation 730, the noise component is converted into a high frequencynoise component (referred to herein as a “decorrelated noisecomponent”), which is spatiotemporally decorrelated from neighboringpixels, through decorrelation.

In operation 740, an image signal is generated by adding the signalcomponent to the decorrelated noise component.

Accordingly, since low frequency noise correlated with neighboringpixels is converted into random (e.g., high frequency noise)decorrelated noise that is spatiotemporally decorrelated fromneighboring pixels, an image signal from which the low frequency noiseis removed can be obtained.

In operation 750, temporal image noise is reduced by performing motioncompensation filtering on the image signal including the decorrelatednoise component.

That is, low-pass filtering is performed between a pixel value of thecurrent field and a pixel value of the previous field in which motion isestimated. For example, the motion compensation filtering may be givenby

(P(x,y,t))+(P(x−mv _(x) ,y−mv _(y) ,t−1))/2  (1)

where P(x,y,t) is an image signal of the current field, P(x,y,t−1) is animage signal of the previous field, and mv_(x) and mv_(y) are motionvectors of an x component and a y component in a (x, y) coordinatesystem.

Alternatively, decorrelation may be differently performed on luminancenoise and chrominance noise.

In this case, spatiotemporal decorrelation is performed on a chrominancenoise component of the chrominance noise to obtain decorrelatedchrominance noise, and the decorrelated chrominance noise is removedthrough motion compensation filtering. The spatiotemporal decorrelationis performed on a still region of an image signal so as not to have amotion afterimage on a screen.

Accordingly, a temporal frequency and a spatial frequency can beincreased since spatiotemporal decorrelation is performed on acorrelated noise signal and noise included in an image signal can beeffectively removed since temporal filtering is performed on adecorrelated noise signal.

FIG. 8A is a flowchart illustrating a decorrelated noise signalprocessing process of a method, of which an example is shown in FIG. 7,according to an exemplary embodiment of the present invention.

In operation 815, it is determined whether a random binary number is “0”or “1”.

If it is determined in operation 815 that the random binary number is“0”, the process proceeds to operation 820. In operation 820, a “0”noise level value is output. Otherwise, if it is determined in operation815 that the random binary number is “1”, the process proceeds tooperation 830. In operation 830, a current noise level value is output.

FIG. 8B is a flowchart illustrating a decorrelated noise signalprocessing process of a method, of which an example is shown in FIG. 7,according to another exemplary embodiment of the present invention.

In operation 845, an image is input through a digital camera or abroadcast channel.

In operation 850, motion information is extracted by using a differencebetween an image signal of a previous field and an image signal of acurrent field.

In operation 860, it is determined whether an image signal is a stillimage by using the motion information.

If it is determined in operation 860 that the image signal is a stillimage, the process proceeds to operation 865. In operation 865, a randombinary number is generated and it is determined whether the randombinary number is “0” or “1”.

If it is determined in operation 865 that the random binary number is“0”, the process proceeds to operation 875. In operation 875, achrominance noise level value of the current field is output. Otherwise,if it is determined in operation 865 that the random binary number is“1”, the process proceeds to operation 870. In operation 870, achrominance noise level value of the previous field is output.

Otherwise, if it is determined in operation 860 that the image signal isa moving image, the process proceeds to operation 875. In operation 875,the chrominance noise level value of the current field is output.

Another exemplary embodiment of the present invention includescomputer-readable codes for performing a method as describedhereinabove, whereby the computer-readable codes are stored on acomputer-readable recording medium. The computer-readable recordingmedium may be any data storage device that can store data which can bethereafter read by a computer system. Examples of the computer-readablerecording medium include read-only memories (ROMs), random-accessmemories (RAMs), CD-ROMs, magnetic tapes, floppy disks, and optical datastorage devices. The computer-readable recording medium can bedispersively installed in a computer system connected to a network, andstored and executed as a computer-readable code in a distributedcomputing environment.

While the present invention has been particularly shown and describedwith reference to exemplary embodiments thereof, the embodiments andterms have been used to explain the present invention and should not beconstrued as limiting the scope of the present invention defined by theclaims. Accordingly, it will be understood by those of ordinary skill inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present invention asdefined by the following claims.

What is claimed is:
 1. A method of removing image noise, the methodcomprising: separating an input image signal into a signal component anda noise component; converting the noise component into a decorrelatednoise component that is spatially decorrelated from neighboring pixels;generating an out image signal that is decorrelated from noise by addingthe decorrelated noise component to the signal component; and performingtemporal filtering on the out image signal to remove temporal imagenoise.
 2. The method of claim 1, wherein the converting of the noisecomponent into the decorrelated noise component comprises adjusting again of a low frequency noise component input according to a randomsignal and converting the low frequency noise component into a randomnoise component.
 3. The method of claim 1, wherein the converting of thenoise component into the decorrelated noise component comprises randomlyconverting a chrominance noise level of a current field into achrominance noise level of a previous field.
 4. The method of claim 1,wherein the converting of the noise component into the decorrelatednoise component comprises: performing a first decorrelation on luminancenoise; performing a second decorrelation on chrominance noise;converting a luminance noise component of a current field into a highfrequency random noise component; and converting a chrominance noiselevel of the current field into a chrominance noise level of a previousfield.
 5. The method of claim 1, wherein the converting of the noisecomponent into the decorrelated noise component comprises: extractingimage motion information by using a difference between an image signalof a previous field and an image signal of a current field; determiningwhether the input image signal is a still image by using the imagemotion information; generating a random signal if it is determined thatthe input image signal is a still image; and converting a chrominancenoise level of the current field into a chrominance noise level of theprevious field according to the random signal.
 6. The method of claim 1,wherein the performing of temporal filtering comprises performingmotion-compensated low-pass filtering on the output image signal.
 7. Themethod of claim 1, wherein the performing of temporal filteringcomprises performing low-pass filtering between a pixel value of acurrent field and a pixel value of a previous field in which motion isestimated.